Enhanced Fake Image Localization in Social Media using Swin Transformer and EfficientNet Feature Fusion

Abstract

Spurious images on social media result in misinformation, online fraud, and manipulation. Conventional CNN-based techniques fail to precisely localize tampered areas because of poor feature extraction abilities and limited generalization. In this paper, the authors suggest a hybrid deep learning method combining CNN with ResNet-50 for enhanced fake image localization. ResNet-50 improves feature extraction through capturing long-distance dependencies and optimal feature propagation, minimizing loss of information. The innovation of this method is efficient spatial and contextual feature fusion through ResNet-50, resulting in better segmentation of fake regions and enhanced adversarial robustness. The model is trained on publicly released datasets, including Kaggle Deepfake Dataset and FaceForensics++, and tested against state-of-the-art localization methods. Experimental results indicate better detection accuracy, better segmentation of tampered regions, and adversarial robustness. This method further improves digital forensics through a robust, effective, and economical means of identifying and locating counterfeit images on social media.

Introduction

The widespread use of deception images on social media poses significant challenges related to misinformation, fraud, and public manipulation. Advances in AI and deep learning have made it easier to create highly realistic fake images, including deepfakes, which are difficult to detect and localize precisely. Traditional handcrafted and CNN-based detection methods lack the feature extraction capabilities and generalization needed for accurate fake image localization.

This research proposes a hybrid deep learning approach combining CNN with ResNet-50 to enhance the detection and localization of manipulated regions in images. ResNet-50’s residual learning architecture allows better global and local feature extraction, minimizing issues like vanishing gradients and improving generalization. Additionally, a feature fusion technique integrates spatial and contextual information, leading to more precise segmentation of fake regions.

The model is trained and evaluated on open-source datasets such as Kaggle Deepfake Dataset and FaceForensics++, utilizing image preprocessing and augmentation to improve robustness and generalization. Comparative analyses show that this approach outperforms existing methods, especially in identifying subtle manipulations and resisting adversarial attacks.

The study highlights the importance of such detection tools for combating misinformation on social media and enhancing digital trust. It also stresses the need for continuous development of adaptive detection models to keep pace with evolving AI-based image manipulation technologies. The proposed method offers a cost-effective, accurate, and practical solution for forensic and fact-checking applications.

Conclusion

The research effectively created a state-of-the-art deep learning method for fake image localization by combining CNN with ResNet-50. The suggested model showed considerable improvements compared to conventional CNN-based methods in detection accuracy, localization accuracy, and adversarial robustness. The feature fusion method successfully merged spatial and contextual information, allowing the model to identify real and manipulated areas with greater sensitivity. Leveraging ResNet-50\'s powerful feature extraction capacity, the system preserved long-range dependencies, promoting best feature propagation and minimizing information loss during training. The improvements promoted better fake region segmentation, enhancing interpretability and robustness. Testing on public datasets, such as Kaggle Deepfake Dataset and FaceForensics++, validated the model\'s generalization across various manipulation methods.The suggested system outperformed the traditional CNN architectures in the detection of subtle and sophisticated forgeries, filling significant limitations of current fake image localizing approaches. The enhanced segmentation accuracy also confirmed the model\'s ability to demarcate manipulated regions more accurately, rendering it a useful application for digital forensics and misinformation control. The research also investigated the model\'s resilience towards adversarial manipulations, where the system proved to remain highly accurate in localization even when perturbed deliberately. This resistance guarantees practicality in real-world applications, specifically in deepfake detection and AI-generated forgery detection.

References

[1] G. S. Jhun et al., \"A New Wave of Texture Feature: Enhancing Deepfake Detection via Image Waveform,\" 2024 15th International Conference on Information and Communication Technology Convergence (ICTC), Jeju Island, Korea, Republic of, 2024, pp. 234-238, doi: 10.1109/ICTC62082.2024.10827484. [2] S. Husseinil and J. -L. Dugelay, \"RAW Data: A Key Component for Effective Deepfake Detection,\" ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Hyderabad, India, 2025, pp. 1-5, doi: 10.1109/ICASSP49660.2025.10887800. [3] S. Concas, G. Perelli, G. L. Marcialis and G. Puglisi, \"Tensor-Based Deepfake Detection in Scaled and Compressed Images,\" 2022 IEEE International Conference on Image Processing (ICIP), Bordeaux, France, 2022, pp. 3121-3125, doi: 10.1109/ICIP46576.2022.9897606. [4] B. Ghita, I. Kuzminykh, A. Usama, T. Bakhshi and J. Marchang, \"Deepfake Image Detection Using Vision Transformer Models,\" 2024 IEEE International Black Sea Conference on Communications and Networking (BlackSeaCom), Tbilisi, Georgia, 2024, pp. 332-335, doi: 10.1109/BlackSeaCom61746.2024.10646310. [5] J. John and B. V. Sherif, \"Comparative Analysis on Different DeepFake Detection Methods and Semi Supervised GAN Architecture for DeepFake Detection,\" 2022 Sixth International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC), Dharan, Nepal, 2022, pp. 516-521, doi: 10.1109/I-SMAC55078.2022.9987265. [6] V. Rogovoi, V. M. Korzhuk and O. A. Kokorina, \"Development of a Deepfake Detection Method: Application of Frequency Analysis and Reduction of the Image Color Space to Improve Classification Accuracy,\" 2024 V International Conference on Neural Networks and Neurotechnologies (NeuroNT), Saint Petersburg, Russian Federation, 2024, pp. 36-39, doi: 10.1109/NeuroNT62606.2024.10585373. [7] P. P and G. R. S, \"Detection of Deepfake Medical Images Based on Spatial and Frequency Domain Analysis,\" 2024 IEEE 16th International Conference on Computational Intelligence and Communication Networks (CICN), Indore, India, 2024, pp. 611-617, doi: 10.1109/CICN63059.2024.10847427. [8] A. Mishra, A. Bharwaj, A. K. Yadav, K. Batra and N. Mishra, \"Deepfakes - Generating Synthetic Images, and Detecting Artificially Generated Fake Visuals Using Deep Learning,\" 2024 14th International Conference on Cloud Computing, Data Science & Engineering (Confluence), Noida, India, 2024, pp. 587-592, doi: 10.1109/Confluence60223.2024.10463337. [9] Z. Deng, B. Zhang, S. He and Y. Wang, \"Deepfake Detection Method Based on Face Edge Bands,\" 2022 9th International Conference on Digital Home (ICDH), Guangzhou, China, 2022, pp. 251-256, doi: 10.1109/ICDH57206.2022.00046. [10] A. Berjawi, K. Samrouth and O. Deforges, \"Optimization of DeepFake Video Detection Using Image Preprocessing,\" 2023 Fifth International Conference on Advances in Computational Tools for Engineering Applications (ACTEA), Zouk Mosbeh, Lebanon, 2023, pp. 59-63, doi: 10.1109/ACTEA58025.2023.10193954. [11] Y. Wang and G. Liao, \"Deepfake Video Detection Based on Image Source Anomaly,\" 2024 IEEE 2nd International Conference on Image Processing and Computer Applications (ICIPCA), Shenyang, China, 2024, pp. 397-401, doi: 10.1109/ICIPCA61593.2024.10709022. [12] Y. Tian, W. Zhou and A. U. Haq, \"Detection of Deepfakes: Protecting Images and Vedios Against Deepfake,\" 2024 21st International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), Chengdu, China, 2024, pp. 1-6, doi: 10.1109/ICCWAMTIP64812.2024.10873771. [13] V. Dudykevych, H. Mykytyn and K. Ruda, \"The concept of a deepfake detection system of biometric image modifications based on neural networks,\" 2022 IEEE 3rd KhPI Week on Advanced Technology (KhPIWeek), Kharkiv, Ukraine, 2022, pp. 1-4, doi: 10.1109/KhPIWeek57572.2022.9916378. [14] H. Vajpayee, N. Yadav, A. Raj and S. Jhingran, \"Detecting Deepfake Human Face Images Using Transfer Learning: A Comparative Study,\" 2023 IEEE International Conference on Contemporary Computing and Communications (InC4), Bangalore, India, 2023, pp. 1-5, doi: 10.1109/InC457730.2023.10263216. [15] H. Wang, Z. Liu and S. Wang, \"MIFAE-Forensics: Masked Image-Frequency AutoEncoder for DeepFake Detection,\" ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Hyderabad, India, 2025, pp. 1-5, doi: 10.1109/ICASSP49660.2025.10889125. [16] A. Zhalgasbayev, T. Aiteni and N. Khaimuldin, \"Using the CNN Architecture Based on the EfficientNetB4 Model to Efficiently Detect Deepfake Images,\" 2024 IEEE AITU: Digital Generation, Astana, Kazakhstan, 2024, pp. 14-19, doi: 10.1109/IEEECONF61558.2024.10585385. [17] M. Mohzary, E. Basunduwah, S. Song and B. -Y. Choi, \"READFake: Reflection and Environment-Aware DeepFake Detection,\" 2024 IEEE International Conference on Visual Communications and Image Processing (VCIP), Tokyo, Japan, 2024, pp. 1-5, doi: 10.1109/VCIP63160.2024.10849940. [18] J. Zhang, K. Cheng, G. Sovernigo and X. Lin, \"A Heterogeneous Feature Ensemble Learning based Deepfake Detection Method,\" ICC 2022 - IEEE International Conference on Communications, Seoul, Korea, Republic of, 2022, pp. 2084-2089, doi: 10.1109/ICC45855.2022.9838630. [19] J. Ding, J. Liu, J. Shi, X. Hu, X. Qiao and H. E, \"DDEM: Deepfake Detection Enhanced Model for Image Forgery Detection Combat Academic Misconduct,\" 2024 11th International Conference on Behavioural and Social Computing (BESC), Harbin, China, 2024, pp. 1-6, doi: 10.1109/BESC64747.2024.10780724. [20] D. Garg and R. Gill, \"Deepfake Generation and Detection - An Exploratory Study,\" 2023 10th IEEE Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON), Gautam Buddha Nagar, India, 2023, pp. 888-893, doi: 10.1109/UPCON59197.2023.10434896.

Copyright

Copyright © 2025 Dr. K. Sivaraman, Md Saif , Mohamed Fahath MR, Mohammed Moosa Kader P S. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.